Skip to content

Dev#33

Merged
zTgx merged 10 commits intomainfrom
dev
Apr 10, 2026
Merged

Dev#33
zTgx merged 10 commits intomainfrom
dev

Conversation

@zTgx
Copy link
Copy Markdown
Member

@zTgx zTgx commented Apr 10, 2026

No description provided.

zTgx added 10 commits April 9, 2026 22:39
- Replace NavigationStep with ReasoningStep that includes detailed
  decision reasoning, stage information, and candidate alternatives
- Add reasoning_chain field to PipelineContext and RetrieveResponse
- Implement reasoning recording for each retrieval stage (Analyze,
  Plan, Search, Evaluate)
- Include decision rationale, strategy used, and LLM call summaries
  in reasoning steps
- Enable auditable and explainable retrieval decisions instead of
  opaque navigation traces

docs: add function specification and scenario documentation

- Create design document outlining core differentiation features
  including structured reasoning chain, query planner, multi-hop
  reasoning
- Add comprehensive scenario documentation covering various use
  cases like AI coding assistant, enterprise knowledge base, legal
  contract review, academic research
- Define API design principles with two-method interface approach
- Document implementation priorities from P0 to P6 with clear
  dependencies

refactor(retrieval): enhance pipeline context with reasoning capabilities

- Update PipelineContext to use ReasoningChain instead of
  navigation_trace
- Add push_reasoning_step and record_reasoning helper methods
- Modify stage implementations to populate reasoning information
- Update type exports to include new reasoning-related types
…nagement

- Introduce RetrievalBudgetController to track token consumption across all
  pipeline stages (Plan, Search, Evaluate)
- Add BudgetStatus enum with Healthy, Constrained, and Exhausted states
- Implement budget-aware strategy selection in Plan stage
- Apply dynamic beam width adjustment based on budget status in Search stage
- Enable graceful degradation when budget is exhausted
- Track and report budget utilization throughout the pipeline
- Add comprehensive tests for budget controller functionality
Add optional path_constraint field to SubQuery to store structural
path constraints extracted from queries (e.g. "3.2", "Chapter 5").

Add resolved_path_hints field to PipelineContext to store node IDs
extracted from structural queries.

Implement Chinese number parsing utility function to convert
Chinese numerals to integers (e.g. "三" → 3, "二十一" → 21).

Add extract_structure_hints method to recognize and map various
structural patterns including:
- Chinese chapter/section patterns: "第X章", "第X节", "第一章"
- English section numbers: "Section 3.2", "section 4.1.2"
- Chapter references: "Chapter 5", "chapter 10"
- Bare section numbers: "3.2.1", "2.1"
- Table/Figure references: "表3", "Table 5", "图2", "Figure 4"

Integrate structure hints into search stage by injecting them as
high-priority search cues with confidence score of 1.0.

Update all relevant initialization code to include the new fields
in tests and implementation.
- introduce ReasoningIndex and related types (TopicEntry, SummaryShortcut,
  SectionSummary, HotNodeEntry, ReasoningIndexConfig) for pre-computed
  topic-to-path mappings
- create ReasoningIndexStage that builds the reasoning index after
  enrich stage and before optimize stage
- add reasoning_index field to PipelineOptions and IndexContext to
  configure and store the reasoning index
- implement hot node tracking functionality in HotNodeTracker for
  identifying frequently retrieved nodes
- integrate reasoning index into persist stage to save it with documents
- update metrics to track reasoning index build time and statistics
- add reasoning cache integration in retrieval pipeline context
- Introduce DocumentGraph struct with nodes, edges, and keyword indexing
- Implement graph builder for computing relationships based on shared keywords
- Add graph-aware retrieval strategy with boost capabilities
- Support serialization and storage integration in workspace
- Include comprehensive test coverage for graph operations
…vent support

This commit introduces comprehensive streaming retrieval capabilities to the
vectorless engine. Users can now receive incremental results through a
channel-based event system that provides real-time feedback during the
retrieval process.

Key changes include:

- Added `retrieve_streaming()` method to `PipelineRetriever` that returns
  a channel for receiving `RetrieveEvent`s as the pipeline progresses
- Implemented `RetrieveEvent` enum with variants for different pipeline
  milestones including Started, StageCompleted, NodeVisited, ContentFound,
  Backtracking, SufficiencyCheck, Completed, and Error
- Created new streaming module with proper channel handling and event
  emission throughout the retrieval pipeline
- Added streaming example in `examples/rust/streaming.rs` demonstrating
  how to process events as they arrive
- Updated `RetrievalOrchestrator` to support both regular and streaming
  execution paths without affecting existing functionality
- Enhanced documentation with usage examples and comprehensive event
  descriptions

The streaming API maintains backward compatibility - existing `query()`
method remains unchanged while `query_stream()` provides the new
incremental functionality. This enables use cases like interactive Q&A
with real-time feedback, progress monitoring, and responsive UI updates.
Removed the main README.md and design documents for client-module
and content-aggregation as they are no longer relevant to the
current codebase structure.

BREAKING CHANGE: Documentation files removed as part of cleanup
process.
…ontent

- Replace old logo with new lovable-vectorless.png and update styling
- Change title to "Reasoning-native Document Intelligence Engine"
- Update crates.io badge to show version instead of downloads
- Remove detailed architecture diagram and how-it-works sections
- Simplify quick start with streamlined Python and Rust examples
- Consolidate API usage patterns into concise code snippets
- Remove configuration details and focus on essential setup
- Streamline feature descriptions and remove redundant sections
- Remove architecture diagram file as it's no longer referenced
@zTgx zTgx merged commit 228f4d8 into main Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant